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Abstract 

The assumption of log-concavity is an attractive and flexible nonparametric shape 
constraint in distribution modelling. In this work, we study the maximum likelihood 
estimator (MLE) of a log-concave probability mass function. We show that the MLE 
is strongly consistent and derive pointwise asymptotic theory, which is used to calculate 
confidence bands for the true probability mass function. The proposed estimator and 
associated confidence bands may be easily computed using the R package logcondiscr. 
We illustrate the fiexibility of the estimator on recent data from the HlNl pandemic in 
Ontario, Canada. 

Keywords: nonparametric estimation, shape-constraints, confidence interval, HlNl, discrete 
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1 Introduction 

Nonparametric maximum likelihood estimation of a log-concave probability density in the 
continuous setting has attracted considerable attentior i over th e last f ew years. The lis t 
of references is extensive, and we refer the reader to IWaltherl ( 20091 ) : ICule et al.l ( 20ld ): 
Seregin and Wellnerl ( 2010l ) and the references therein for an overview of recent theoretical and 
computational developmen ts. The merits of using l og-concavity as a s hape coii s traint have 
been discussed in detail in iBalabdaoui et al.l (J2009|) , ICule et all (|2010l ) , IWaltheri (|2009l ) , and 



Diimbgen and RufibachI (J2011I ). Not only do many parametric models admit a log-concave 
density, but log-concavity is also a valuable surrogate for unimodality Indeed, log-concavity 
implies unimodality, and whereas the nonparametric MLE of a unimodal density does not exist 
(see e.g. iBirgd 119971 ). it does exist under the assumption of log-concavity Furthermore, a log- 
concave density is the natural generalization of the normal density, where log/(x) = — x^/2. 
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the "canonical" concave function. However, log-concavity gives more flexibility in modelling 
since it allows for asymmetry. It also allows for a wide-range of tail-behaviour, with the 
slowest being that of the double-exponential where log/(x) = — |x|. 

Given the large corpus of work on estimation of a log-concave density in the continuous 
setting, it comes as a surprise that little attention has been given to estimati on of a log- 
conca ve probability mass function (pmf). The unpublished Master's thesis of IWeyermann 
( 20081 ) is the only work of which we are aware. As in the continuous setting, the assump- 
tion of log-concavity is highly appealing. Many discrete parametric models admit a log- 
concave pmf. Binomial, negative binomial, geometric, hyper geom etric, uniform, Poisson, 
hyper-Poisson (JBardwell and Growl. ll964l:ICrow and Bard welll. Il965l). the Polya-Eg genberger, 
and the Skella m distribution (lKa.rlis and Ntzoufras . 2006; Alzaid and Omaiij . l2010l ) are some 
examples; see Ijohnson and Kota ( 19691 ) and iDevroy e (19871) for further details. Therefore, 
the log-concave assumption provides a broad and flexible, yet natural, non-parametric class of 
distributions on Z. In Section r2.lt we give a formal definition of log-concavity of a pmf. The 
motivation behind this formalism is to make the link between the different definitions given 
in the literature, and also to avoid some pitfalls as illustrated in a counterexample given in 
(12. ip . In addition, as we show in Section [2.21 if data admitting a continuous log-concave den- 
sity has been grouped, then the resulting pmf is also log-concave. Therefore, the log-concave 
pmf provides an appropriate surrogate for many data problems where the log-concave density 
assumption would have been used, but where the data has been grouped or discretised. 

In FigurelH we show an example of the nonparametric MLE under the log-concave assumption. 
The true underlying distribution is the negative binomial with parameters r = 6 and p = 0.3. 
The behaviour of the MLE is considerably better than that of the empirical mass function; 
the MLE is able to smooth out the rather erratic empirical pmf and recover the general 
shape of the true pmf quite well, even for these small sample sizes. Further simulations are 
shown in Section [H where we study the finite sample performance of the proposed estimator 
via simulations. Here, we find that the MLE performs well, even when compared to the 
parametric estimator, but more so when the logarithm of the true pmf does not have steep 
slopes. This behaviour can be further explained by the theoretical results of Section HI 
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Figure 1: Nonparametric MLE of the negative binomial (6, 0.3) distribution for n = 25 (left) 
and n = 100 (right). 



In IWevermannI ( 20081 ) . it was shown that the MLE of a log-concave pmf on Z, exists and is 



unique. Furthermore, the logarithm of the MLE is piecewise Unear and we use the term knot 
point to describe any point where a piecewise hnear function has a bend. As in the con- 
tinuou s case, the MLE admits knots at some subset of the set of observations. IWevermann 



(120081') also pro v ides a n active set algorithm to compute the MLE, much in the spirit of 
Diimbgen et al.l (l20ld). which was i mplemented in Matlab. We hav e adapted this code t o 
R ( R Development Core Teaml . I2OIII ) in the new package logcondiscr ( Rufibach et a l.U2011). 
available from CRAN. In Section [3l we recall some of the main results of I Wever man n (200q), 
and provide additional characterisations of the estimator. These characterisations are im- 
portant in the study of the estimator and provide crucial insight into its behaviour. In this 
section, we also give some details on the computation of the MLE. 

In Section HI we study the asymptotic behaviour of the MLE. Our main theoretical results 
establish strong consistency and robustness of the MLE, as well as its pointwise weak conver- 
gence on any segment between two successive knot points of the logarithm of the true pmf 
which are a finite distance apart. For our theory of weak convergence to hold, we require that 
the true log-concave pmf admits a one-sided support. Note that this assumption is not too 
restrictive as it is satisfied by most of the known parametric log-concave discrete distributions. 
Unlike the continuous setting in R, which admits rates of r?'^ for a twice differentiable density, 
here convergence occurs at the parametric rate ^Jn. The limit distributions of the MLE, its 
logarithm, and the corresponding cumulative distribution function (cdf) are explicitly given. 
These limits are characterised in terms of an en velope-type process , M, w hich can be viewed 
as a discrete analogue of the random process of iBalabdaoui et al.l ( 20091 ) , also appearing in 



the poi ntwise convergence of the MLE of a convex decreasing density; see iGroeneboom et al. 
(J2001bl ). We also show that the limiting process can be described as the solution of an 



appropriate least squares concave regression problem. The advantage of this approach is that 
the solution o f the least squares prob lem can be found explicitly using, for example, the R 
package cobs (JNg and Maechleij . 120111 ). Therefore, we are able to sample directly from the 
limiting distribution, which allows us to compute pointwise confidence bands for the true pmf. 
The details of our approach are given in Section 14.41 

To our best knowledge, this is for the first time t hat confidence bands hav e been computed 
in the log-concave setting. In the continuous case, IBalabdaoui et al.l (120091 ) derive pointwise 
asymptotics for the MLE. However, these depend on the value of ■0"(2^o)) where ^ = log/, 
which is dif ficult to estimate. Sim il ar pr oblems arise for the monotone shape-restriction, and 
the work of iBaneriee and Wellnerl (|200ll ) was designed to overcome this issue. Currently, no 
such results exist for the log-concave MLE on M. 

As an illustration of our methods, we apply the proposed estimator to a real data set of 
HlNl influenza p andemic data froin Ontario, Canada. This data comes from an early study 
of the pandemic, iTuite et al.l (J20ld ). when it was crucial to provide a quick analysis of the 
behaviour of the virus. The flexibility of the log-concave assumption makes it suitable to 
describe important aspects of the incubation period of the swine flu, as well as the duration 
of symptoms. To handle errors in the data collection, we use a simple mixture model, which 
gives even more flexibility to our approach. 

Conclusions and a discussion can be found in Section [71 Appendix A provides some notation 
which is used throughout this paper, and Appendix B discusses some of the properties of 
log-concavity. There, we also give some classical examples of pmfs for which we verify log- 



concavity and give the explicit location of the corresponding mode when possible. All proofs 
and technical details have been collected in Appendix C. 



2 Log— concavity of discrete distributions 

Our first goal is to give a precise definition of log-concavity in the discrete setting. Although 
such a property might seem to be obvious to describe mathematically, we find that it is 
necessary to give a formal definition to avoid certain pitfalls. Some further details of log- 
concavity are provided in Appendix B. In particular, we give a definition of log-concavity 
based on that of concavity. To our surprise, no reference of concavity in the discrete setting is 
available, and hence o ur effort to fill this gap in S ection 17. 1[ Beyond what is provided below , 
we refer the reader to iKeilson and Gerbed ( 197ll ) and iDharmadhikari and Joag-DevI ( 19881 ) 
for further details on log-concavity in the discrete setting. 



2.1 Definition of log— concavity 

Recall that the definition of log-concavity is closely tied with that of unimodality. This is true 
for densities as well as probability mass functions. Consider then a pmf p = {pk, k £ Z}. 

Definition 2.1. (Unimodality) There exists at least one km G ^ such that 

Pk > Pk~i for all k <km 
Pk+i < Pk for all k>km. 

The integer km in the definition above does not have to be unique. In case there exist several 
integers satisfying the same property, the smallest one will be defined as the mode of p. 

Definition 2.2. (Strong unimodality) p is strongly unimodal if for any unimodal pmf q, the 
convolution p-k q is again a unimodal pmf 



As indicated bv lKeilson and Gerbeij (|l97ll ) strong unimodality implies unimodality since the 
Dirac distribution at zero is unimodal. In their Theorem 3, they prove equivalence between 
strong unimodality and log-concavity. 

Definition 2.3. (Log-concavity) The pmf p is log-concave if both of the following conditions 
hold: 

A. If i < j < k are such that PiPk > 0, then pj > 0. 

B. pI > Pk-iPk+i for all k eZ. 



The definitions above extend of course to nonnegative functions which are not necessarily 
probability mass functions. In Definition 12. 3( Property A says that the support 5 of p is of 
the form {ki, ki + 1, ..., k2 — 1, ^2} where ki < k2, ki G ZU {—00} and ^2 £ZU{+oo). Other 



equiva lent descriptions were proposed in the literature. For instance, iKeilson and Gerber 
(I1971I ) express the same property in terms of conne ctivity (equiva lent here to convexity) of 
the support of p. The second description is due to IStanlevi (19891) which requires that the 



sequence {...,p-i,pQ,pi, ...} does not have an internal zero. Property A is far from being 
superfluous, as illustrated in the following example, given to us by Marios Pavlides in a 
private communication. Consider the pmf p such that 

po = l/2,pi=p2 = 0,P3 = l/2, and pfc = for A; eZ\ {0,1,2,3}. (2.1) 

Although p does satisfy Property B, it is easy to see that it is not u r iimod al, and hence it 
is not strongly unimodal or equivalently not log-concave. In IPevroyd (119871 ). a definition of 
log-concavity based on Property B only is given. Unless it is explicitly said that the support 
of the log-concave sequence is equal to Z, the counterexample in ()2.ip indicates clearly that 
Property B alone is not enough. 

The next proposition gives an equivalent condition for log-concavity of a pmf p. 

Proposition 2.1. A pmf p = {pk,k S Z}, with support S, is log-concave if and only if S is a 
subset in Z of consecutive integers and the sequence {pk/pk-i, k ^ S} is nonincreasing. Here, 
the first term of the ratios is allowed to take the value oo. 

Note that the proposition above gives the possibility of constructing an estimator based on 
"monotonising" the empirical probability ratios. This alternative approach will be pursued 
elsewhere. 

One may also discuss log-concavity in terms of the "slopes" of the function ^lj{k) = \ogpk- For 
any k £ S, (A^)^ < for a log-concave pmf, where {A^)k = ipk+i ~ 2'(/'fc + V'fc-i denotes the 
discrete Laplacian. However, the condition (A^)^ < for fc E Z is not sufficient to check for 
log-concavity of a pmf, as (A'^)^ may not be well-defined. For this reason, we provide more 
exact conditions on concavity in Appendix B. 

Definition 2.4. For any concave function f with 5 = {/c G Z : (p{k) > — cxd}, we call any 
point k £ S such that {Aip)k < a knot point of the function ip. Here, we use the convention 
a — oo < for any real number a. Furthermore, if both k and A; -|- 1 are knots, then we say 
that k is a double knot, and if k — l,k and k + 1 are knots, then we say that k is a triple knot. 

2.2 Grouped versus continuous data 

In this section, we establish the link between continuous and discrete log-concave distribu- 
tions. Let {Ai,i £ Z} denote a partition of of the positive real line such that \Ai\ is constant. 
We assume that each interval Ai is either of the form {ai, Pi] or [aj,/3j). That is, we assume 
that each interval is either right -open and left-closed, or vice versa. For a continuous random 
variable X admitting a density we then define the probability mass function pi = P{X £ Ai). 
Such a scenario arises either under grouping (as in the HlNl example considered in Section [6|) 
or discretisation, where one observes Y = 5\_X/5\ , for example, instead of the continuous ran- 
dom variable X. We have the following result. 

Proposition 2.2. Suppose that the continuous random variable X has a log-concave density 
with respect to Lebesgue measure on R. Then the probability mass function p = {pi} is also 
log-concave. 

Throughout this paper, we focus on the probability mass function defined on Z. However, 
our results are applicable to a pmf defined o n any regul a r grid , as long as that grid does not 



depend on sample size, as was considered in lTang et al.l (J2011 



3 Properties of the maximum likelihood estimator 

3.1 Existence 

Let (Xi, . . . , Xn) denote a random sample from the pmf p. Then, the MLE of a log-concave 
pmf is found by maximising the log- likelihood '^^i^i^og pxi/n, over the class of log-concave 
pmfs, CCi. Let m be the total number of the ordered distinct values ki < ■ ■ ■ < k^ occurring 
in the sample (Xi, ..., X„), X = {/ci, . . . , k^] the set of these observations, and Wj = pn,kj = 
'n~ ^ y^"-i l {Y;=/ir. .}, j = 1, ■■■,m, the corresponding empirical frequencies. By Theorem 3.1 of 



i), 



Silvermanl (|l982l ). the MLE can also be found as the maximiser of ^"1^ Wj log(pfc O^X^feezPfc' 



over the class CC (the class of log-concave nonnegative sequences), where the term 'Ylik&zPk 
is the Lagrange multiplier. Using the one-to-one correspondence between the classes CC and 
the class of all concave functions C, it follows that the MLE exists if and only if the criterion 
function 

m 



admits a maximiser ipn over C. Then, the maximum likelihood estimator is given by pn,k = 
expipnik) for A: S Z. 

Reducing the set of functions over which $„ is maximised is one of the key steps in proving 
existence of ipn- It also sheds more light on the shape of the estimator, and is of crucial 
importance when setting up an algorithm to compute pn- We first introduce the family J>„ 
of functions 

J-^ := {(/; : Zn[A;i,A:^]^IR, (^ = -ooonZn{M\[A:i,A;„]}}. 

For any ip G Trn, we consider the set of knot points JC{(p) = {/c £ Z n [fci, km] ■ iy^)k < 0}. 
Note that ki and km are always in JC{ip). If we consider the piecewise function defined on 
[ki,km] interpolating (p between the points kj,j = l,...,m, then the knot points of f are 
exactly the bending points of its interpolating function; i.e., the points at which this function 
changes its slope. Finally, we consider the sub-family FmiX) = {99 € Fm '■ ^{^) ^ 2^} of 
functions in Fm which only admit knots in the set of observations X = {/ci, ..., km}, and we 
let CmiX) be the subset of concave functions if in FmiX). 

The following results, which we recall here for corn pleteness of exposition, have been shown in 
the unpublished Master's thesis I Weyer manni (120081 ) . A shortened proof is given in Appendix C. 



Theorem 3.1 (IWeyermannl . l2008l ) . We have the following: 

(i) Maximisation of ^n over C is equivalent to its maximisation over CmiX), 
(a) the maximiser 



ipn ■■= arg max $„((/?) 



exists and is unique. 



Therefore, attention can be restricted to concave functions ip such that (p = — oo outside 
[fei, km\ and having knot points only in the set of observations. If ki, ..., Kp denote the internal 
knot points of ^„, then it is not difficult to see that -^n must have the following form 

^n{k) = a + bk + ^Ci{Ki-k)+, keZn[ki,kni] (3.2) 

where a, 6 G M and Cj < 0. 

Remark 3.1. Note that given the location of the knots as in (j3.2p . to find the MLE one needs 
only find the p + 2 unknown values of a,b,ci, . . . , Cp. Note also that since we define the first 
and the last point of the MLE also as a knot, then the number of unknown values is exactly 
equal to the number of knots of the log-MLE. In addition, from Lemma VJ.^A we know that the 
MLE satisfies p + \ equalities in (|3.4p . plus J2xPn,x = 1- Hence, we have p + 2 equations with 
p+2 unknowns, as long as the locations of the knots are known. In essence, this tells us that 
the "degrees of freedom" of the estimator is equal to the number of knots. We believe that this 
characteristic is one key to the quality of the performance of the MLE, as compared to, for 
example, the empirical estimator, which has higher degrees of freedom. Further com,parisons 
are provided in Section\^ We also make use of this heuristic when we develop our confidence 
bands in Section\4.4\ 



3.2 Characterisation 

In the study of shape-constrained estimators, characterisations provide invaluable insight into 
their behaviour. These are often referred to as the Fenchel conditions, due to their relationship 
with Fenchel duality in convex optimization problems. The characterisation of the MLE of a 
log-concave pmf is given b elow. Note that it shares a lot o f similarity with the characterisation 



in the continuous setting (JDiimbgen and Rufibachl . |2009| . Theorem 2.4). In what follows, F„ 



denotes the empirical cumulative distribution function. 

Lemma 3.2. Let ip G Cm(2^) such that Fn{x) = Ylk=k exp^(A;), x G Zn [ki,km] satisfies 
Fn{km) = 1- Then, ip = ipn if o,nd only if the following conditions hold 

jx-l X-l 

Y,Mkj){kj+i-k,) + ¥nikjJ{x-kjJ > J^FnW, V x G Z n [fci, A:„] (3.3) 

j=l k=ki 

x-l 
= y Fn{k), if X is a knot of tp (3.4) 

k=ki 

where jx is the unique index such that kj^ < x < fej^+i. We use the conventions that both 
sums are equal to if x = ki, and that jx = m if x = km ■ 

The above characterisation allows us to uniquely identify the estimator in terms of simple in- 
equalities. Furthermore, the linearity of the characterisation allows us to establish asymptotic 
theory for the estimator in Section HI However, this characterisation is by no means unique, 
as there are other ways of uniquely identifying the MLE. Further properties of the MLE are 



given in Proposition 17.11 in Appendix C. As an immediate corollary to Proposition 17.11 we 
obtain the following identity and bounds, where the latter hold for any a G M and m > 1. 



/ J XPn,x — / ^ XPn^: 



E 



X — a\ Pr, 



< 



E 



X CL\ Pn,x- 



(3.5) 
(3.6) 



Hence, the MLE has the same mean as the empirical distribut ion and a smaller variance than 
the empirical d i stribu tion. Similar bounds were observed in iDiimbgen and RufibachI ( 20091 ) 
and lCule et al.l (2010|) for the MLE of a continuous log-concave density. 



3.3 Computation of the MLE 

Below, we provide onl y a brief desc r iption of how to compute the MLE of a log-concave pmf. 
For details we refer to IWevermannI ( 20081 ). First, any function ip G J^m{T) can be identified 
with the vector ip = {ip{kj))'JLi ^ ^"^- Conversely, each vector i/> E M™ defines the function 
ip £ Fm{Z) via 



^(z) := (l-{z-kj)/5j^ijj + {z-kj)/5j 



for z G Z n [kj, kj+i], 5j = kj^i — kj and 1 < j < m. Using this representation and using the 
linearity of ^ between kj and kj^i we can write the Lagrange term of I as 

m-lSj-l 

m—l oj m— 1 

j=l 1=0 j=2 

m—l m—l 

= J2 "^^1 ^^^' ^'=+1) ~ Y ^^p ^j 

i=i i=2 

where J5. {ipj , ■0i+i) = Sjio exp H 1 - i/Sj)i{jj + (i/5j)V'j+i j , for j E {1, . . . , m - 1}. Note the 
similarity of the Lagr ange term to the corresponding expression in the continuous case, see 
Diimbgen et al.l ( 20ld . Section 2). The rewritten log-likelihood function (|3.1|) that we seek to 
maximise then amounts to 



m—l 



m—l 



'^nW 



j=l j=l j=2 



which is now a concave function M™" — )• M. Furthermore, a function ip £ Tm{Z) belongs to 
Cm{Z) if and only if the corresponding vector i/? E M™ meets the following conditions: 






> 



V'i+i - i^j 



for j = 2, . . . ,m — l so that we end up wit h the task of maximizin g; the function $„ : M™ — )• M 
subject to m — 2 hnear constraints. As in lDiimbgen et alj (|201Q ) an active set algorithm can 
then be set up to solve this maximization problem. 



After streaml ining it, we have translated the or iginal Matlab code developed in IWevermann 
( 2008) to R (R Development Core Teaml . 1201 ll ) and provide in the R package logcondiscr 



( Rufibach et al. 



2011 



the function logConDiscrMLE that computes the MLE from a given 
sample. The package also provides functions to compute the pointwise confidence bands 
introduced in Section 14.21 and the fc-inflated log-concave pmf as discussed in Section [6l 

R offers built-in functions to generate samples of random numbers from standard distributions 
(Binomial, negative Binomial, Geometric, Poisson ) . Correspon ding functions for the Skellam 
distribution are available in the R package skellam ( Lewisl . 12003 ). To generate ran dom numbers 
from an arbitrary log-concave pmf, the algorithm discussed in iDevrovd ( 19871 ) can be used. 
The generator is a special case of a rejection sampling algorithm and uses the fact that for 
any log-concave pmf with a mode at m and probabilities pk we have 

Pm+k < min{pm,Pmexp(l -pml^l)}, 



for all k. For a corresponding result for a log-concave density function see e.g. iRufibachI (12006 
Lemma 2.2.1). 



4 Consistency and ^/n asymptotics of the MLE 

4.1 Consistency of the MLE 

For two probability mass functions on Z, say p and q, we define 



PKhiqWp) 






Pk 



to denote the Kullback-Leibler (KL) divergence, also known as the relative entropy. Recall 
that pkl (q \\p) > 0, however, pkl {q \\p) 7^ PKL (p \\ q) and hence the KL divergence is not a 
metric, but rather a premetric. Note that pn = argmiUgg^j^^pKL (^||Pn) , where pn denotes 
the empirical pmf, and therefore the KL divergence is a natural measure to consider in the 
context of maximum likelihood estimation. In the following, p denotes the true pmf, and 
ijj = log p. 

Theorem 4.1. Suppose that p is a discrete distribution on Z with finite mean such that 
^^Pfclogpfc < oo. Then there exists a unique log-concave pmf on TL, p, such that 

p = ar gmiug^^ci PKL {q\\p)- 

Furthermore, 7i(j)n,p) -^ almost surely. 

The proof is deferred to the Appendix. Note that by Lemma [7.31 the above convergence holds 
also in all other metrics described there, including pointwise convergence and ik distance for 
any A; > 1. The result tells us that, even if the class CCi was originally miss-specified, the 
MLE converges to the pmf which is closest, in the KL sense, to the true pmf. Of course. 



9 



if p is log-concave, then p = p, and our result implies that the MLE is consistent (note 
th at any log-concave pmf sa tisfies the conditions of the theorem). Our proof follows that 
of ICule and SamworthI (2010|). It also provides an alternative way of showing existence and 



uniqueness of the MLE. 

Corollary 4.2. Let F„(x) = Yly<xPn,y^ o.'^d let F = J2y<xPy ^^^n 



sup|F„(x) -F(x)| -^ 

x£Z 



almost surely. 



Recall Definition 12.41 of a knot point. The following result states that knot points of the ipn 
are also consistent. 

Lemma 4.3. For any knot point r of the logarithm of the pmf p, there exists a positive 
integer no sufficiently large such that for all n > tiq, r is also a knot point of the MLEpn with 
probability one. 

Prom a theoretical point of view, Lemma F4.3l is very important in deriving weak convergence of 
our estimator. In practice, it implies that a knot of the logarithm of the true log-concave pmf 
is also a knot of the log-MLE ^jJn when the sample size is large enough. The same lemma does 
not say anything about the converse property, as an observed knot of V'n is not necessarily a 
true knot. The confidence bands derived in this work rely, however, on our knowledge of the 
knot points. What allows us to overcome this issue is Remark 13.11 Namely, assuming more 
knots than necessary only increases our degrees of freedom. In Section 14.41 we discuss further 
the impact of the knot points on confidence bands. 

4.2 Pointwise asymptotics of the MLE 

Let p denote the true log~concave pmf p and V = log p. In what follows, we assume that 
p has a left-sided support. The theory in the case of a right-sided support is deduced from 
the latter using the transformation x i— )• — x which preserves log-concavity. Without loss of 
generality, we assume that the support of p is of the form [0, . . . , a] n M with a G W U {+oo}. 
Fix a point x which lies between two knot points < r < s, that are a finite distance apart. 
This excludes distributions such as the geometric; we consider these to be the "degenerate" 
cases, and these will be studied elsewhere. 

In the following, U denotes a Brownian bridge from (0,0) to (1,0). For x £ {r, . . . ,s — 1}, 
define the quantities 

W„(x) = ^/n{pn,x - Px) W„(x) = \/n{pn,x - Px), 

X X 

Gn(x) = Y,^n{k) Gn{x) = J^W„(fc). 

k=r k=r 

In addition, for x G {r, . . . , s}, we define 

x—l x—1 

Sn(x) = ^ Gn{k) Y„(x) = Y, Gn{k), 

k^r k=r 
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with the convention that ]HI„(r) = Y„(r) = 0. It is well-known that the processes on the right, 
Wn,Gn, and Y„, have Gaussian limits. Finally, let F denote the cdf of the true pmf p. We 
define the least squares (LS) functional 



-. s— 1 s — 1 

^ia) = ^J^5'(2;)Px-J^5(x)w(x), (4.1) 



over the class of concave functions in Ri^''---''^ ^), where W(x) = \J{F{x)) — U{F{x — 1)). 

Proposition 4.4. The functional ^ in (|4.ip admits a unique niinimiser, g* , over the class of 
concave functions on {r . . . , s — 1}. Furthermore, g = g* if and only if the process M defined 
on {r, . . . , s} satisfies 

jj(^J<Y(x), forxe{r,...,s-l} ^^^^^ 

I = Y(x), i/x G {r, . . . , s — 1} is a knot of g* 

where 

x—l k x—l k 

Y(^) = EEw(-?')' "^'^ nx) = Y.Y.9*^j)P3^ (4-3) 

k=r j=r k=r j=r 

with the convention that Y(r) = EI(r) = 0, and the boundary condition Y(s) = EI(s). Note 
that for X £ {r, . . . ,s — 1} 

g [x] = , 

Px 

with the further convention that EI(r — 1) = 0. 

We are now able to state our main asymptotic result. 

Theorem 4.5. Let r < s be two successive knot points of the true pmfp, and let M denote the 
(unique) process on {r, . . . , s} as defined in Proposition \4-4\ with the additional convention 
that M{r - 1) = 0. Then 

V^{Fn{x) - F{x)) A (V]HI),. + U(F(r-l)), 

Vn{Pn,x-Px) -^ (A]HI)a;, 

^M , ^ d (AM). 

Px 

Note that if r = s - 1 in the above that, by definition, (AIHI)^ = B.{r + 1) - 2IHI(r) + m{r - 1) = 
M{s) = W(r). In other words, if the pmfp at x satisfies (Af/;). < and (A'i/').+i < 0, then the 
limiting distribution of \/n{pn^x — Px) is the same as the limiting distribution of \/n[pn^x —Px)- 
In fact, the following stronger statement holds. Similar beh aviour has been observed for the 
discrete Grenander estimator ( Jankowski and Wellneii . l2009l ). 



Corollary 4.6. Let x be a triple knot point of the true pmf Then there exists an uq such 
that for all n > uq 

Pn,x — Pn,x 

almost surely. 
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The situ ation in this discrete settin g shares strong similarities with the on e initially encoun- 
tered by iGroeneboom et alj ( 2001bl ) in convex estimation and afterwards in lBalabdaoui et al 
( 2009 ) in log-concave estimation in the continuous setting. In both works, the limit distribu- 
tion of the nonparametric estimators involve a stochastic process that stays above (invelope) 
or below (envelope) a certain Gaussian process, whose second derivative is convex (concave) 
and upon which depend the limit of the estimators. Knots of this second derivative are touch 
points of the in/en-velope and the Gaussian process. 



4.3 Computation of the limiting process 
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Figure 2: Triangular log-pmf, the functions g* and g*Px and the limit processes H and Y. On the 
bottom left plot ex — W{x)/px- 



To compute H we need to minimise the least squares functional (14.1 



1 "^'^ 



W{x] 



2 ^-^ 1 ■^y-j\ J v . J 2 ^ ' n 

x=r ^^ x—r ^^ 



1 '"^ 



W(x) 



on the class of concave functions on {r, . . . , s — 1}, where W{x) = \]{F{x)) — U(F(x — 1)). 
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Minimising $ is then equivalent to minimising 



s-l 



C{g) = "^Px [g{x 



W(x) 



over the class of concave functions on {r, . . . , s — 1}. This is a weighted least squares concave 
regres sion problem that can b e numerically solved using the function conreg in the R package 
cobs ( Ng and Maechled . 1201 ll ). Note that the vector (W(r), . . . , W(s — 1))"^ is multivariate 



normal with mean and covariance matrix {piSij — piPj)ij=i^^^^^r-s- 

To illustrate Proposition 14.41 we compute g* and the processes H and Y for the triangular 
log-pmf supported on {o, . . . , d} 

a,b,cAe _ f c{x - a)/{b -a) for a; G {a, ... , c} 



^^ \ (e - c)(x - b)/{d -b) + c for a; G {c, . . . , d}, ^ ' 

where we choose a = l,6 = 7, c = 8,(i = ll,e = 2 and normalise the resulting function so that 
the total mass is equal to one. Note that the chosen model also allows us to vary the length 
of a segment in a simple way by suitably defining the parameters o, 6, c, d, e. 

A plot of the log-pmf for the specified parameters is provided in Figure [2] (top left). The plots 
in Figure [2] correspond to the first segment, i.e. r = 1 and s = 7. We provide a plot (top 
right) of the limit oipx as it appears in Theorem [431 namely g* -px (see also Proposition I4.4p . 
a plot of the points W(x) and the corresponding weighted concave regression fit as computed 
by conreg (bottom left), and finally a plot of the Gaussian process Y and its envelope EI on 
the segment {r = 1, . . . , s — 1 = 6}. 

4.4 Pointwise confidence bands 

The main application of the asymptotic results described above is that they may be used to 
calculate pointwise confidence bands for the true pmf. For our theory to apply, we assume 
that the true log pmf has only finite intervals between knot points, thus excluding geometric- 
like distributions. However, these "degenerate" cases form only a small subset of the class of 
log-concave pmfs. Below, we describe how to compute 95% confidence bands, but the method 
can be generalised easily to any other coverage. Furthermore, we describe how to calculate 
the bands over the entire length of the support of the MLE. A similar approach can be also 
used over a smaller subsegment. 

Let S denote the support of the true pmf, and write S = Ujlj, where Ij = {sj, . . . , Sj+i — 1}, 
where the Sj denote the knot points of the true log-pmf. For each x G Ij, let qi{x),q2{x) 
denote 2.5% and 97.5% quantiles of the distribution of (AEI)^;. Since we can simulate directly 
from the distribution of (AIHI)^^^, these are straightforward to estimate. Then, 

{Pn,x + qi{x)/Vn,Pn,x + q2{x)/\/n}, 

give pointwise approximate confidence bands, which are asymptotically correct. Note that 
if \Ij\ = 1, then for x G /j, qi{x) = -l.9Q^Px{l - Px) and q2{x) = +l.QQ^Px{l - Px), by 
Corollary | 
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Figure 3: Estimated 95% quantiles at different points (left) and lengths of confidence intervals 
at one point (right) for the triangular density (J4.4I) with a = 1, 6 = 7, c = 8, e = 2. On the 
left, d = 11 and the quantiles were estimated from the true pmf and assuming different knot 
points, as indicated. On the right, the lengths of the confidence intervals were estimated using 
the MLE at X = 9 for different choices of d > 9. 

To calculate the quantiles qi and (?2 we need to know the true pmf, including the true knot 
points of the log-pmf. The true pmf is easily estimated by the MLE, but a more serious issue 
is that we do knot know the true locations of the knot points. We propose to estimate these 
as the knot points of the log-MLE ipn- As noted following Lemma 14.31 the knot points of tpn 
will, at worst, asymptotically overestimate the set of true knots. We believe that the penalty 
for this is a slight overestimation of the quantiles. Our reasoning relies on Remark 13.11 and 
the following discussion. Overestimating the true set of knots causes us to overestimate the 
degrees of freedom of the estimator, which in turn means that we have overestimated the 
quantiles in the confidence bands. 

An example of the confidence bands for the MLE examples given in Section [H Figure [T] for 
the negative binomial distribution are shown in Figured! The proposed confidence bands are 
shown in red. Confidence bands based on using all knots are shown in blue for comparison. 
Note that the width of the blue bands is the same as the width of pointwise bands for the 
empirical pmf, as they are both based on the same normal approximation. A function to 
compute the confidence bands is available in the R package logcondiscr. 

5 Finite sample performance of the MLE 

The results of the previous sections provide information on the performance of the MLE 
for very large sample sizes. To better understand the behaviour of the estimator for finite 
sample sizes, we have compared the results of the non-parametric vs. the parametric MLE 
for simulations from the Poisson (A = 2) and negative binomial {r = 6,p = 0.3) distributions. 
In each case, we calculated 

1. the empirical pmf (the MLE with no underlying assumptions). 
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Figure 4: Nonparametric MLE of the negative binomial (6, 0.3) distribution for n = 25 (left) 
and n = 100 (right). The empirical distribution is shown in gray, with the MLE in red and 
true pmf in black. The confidence bands are shown as dashed lines, with knots based on the 
MLE in red, and selecting all points as knots in blue. 



2. the log-concave MLE, 

3. the parametric MLE assuming the geometric distribution, 

4. the parametric MLE assuming the Poisson distribution, 

5. the parametric MLE assuming the negative binomial distribution. 

Figures [5] and [6] show boxplots of a variety of distances of each fitted distribution from the 
true pmf. The distances considered are the ii, £21^00, and the Hellinger distance, and each 
boxplot is the result of 1000 simulations. In Figure [5] the sample size is n = 50, and in Figure 
El it is n = 500. 

The power of the non-parametric assumption is clearly shown in these simulations. The log- 
concave MLE performs well in estimating both distributions, albeit not as well as the correct 
parametric MLE. Making an incorrect parametric assumption carries with it the greatest 
cost, and this behaviour is much amplified when sample size is increased. Note, however, that 
the negative binomial MLE performs well for the Poisson distribution. This is because the 
negative binomial converges to the Poisson when p = A/(A + r) and r — )• 00. 

Lastly, the performance of the log-concave MLE is superior to that of the empirical estimator, 
but much more so in the negative binomial setting as compared to the Poisson setting. The 
asymptotic results of the previous section again provide insight into this behaviour. Figure 
[7] shows the pmf as well as its logarithm for both distributions. Note that the Poisson dis- 
tribution exhibits more change in log(p) than the negative binomial. When the true density 
is strictly log-concave. Corollary 14.61 states that the empirical pmf and the log-concave MLE 
are asymptotically equivalent. Because the Poisson distribution is more strictly log-concave 
than the negative binomial, we see this behaviour much sooner in the simulations. In other 
words, the flatter the true ip = log(p) is, the better the performance of the log-concave MLE 
over the empirical distribution. Similar behaviour was noted for the Grenander estimator of 
a decreasing pmf in I Jankowski and Wellneii ( 20091 ). 
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Figure 7: The probability mass function and its logarithm for the Poisson (2) and negative 
binomial (6, 0.3) distributions. 
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6 Example 



We illustrate the new estimator on HlNl influenza data from Canada. iTuite et al.l ( 2010 ) 
report an early study of the HlNl pandemic. The goal of the study was to understand the 
behaviour of the disease, including the incubation period (time from exposure to the disease to 
onset of symptoms) and the duration of symptoms. HlNl individual-level data was collected 
for laboratory-confirmed cases of the disease for a 3-month period in the spring of 2009. From 
these, information on the incubation period (in days, n = 316) and symptom duration (in 
days, n = 712) was derived. For more details on data acquisition we refer to iTuite et al. 
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(|20ld ). 

Clinicians and mathematical biologists are most often interested in the fitted mean, stan- 
dard deviation, and range to understand the behaviour of the virus. These may be used in 
a sensitivity analy sis of the developed deterministic and stochastic models, as was done in 
Tuite et al.1 ( 20ld ). For example, output of these models would be checked against known 



behaviour from the fitted distributions, to ascertain the appropriateness of the former. Al- 
ternatively, the model may use the fitted distribution itself within the algorithm, and hence 
requires the ability to simulate from the fitted distribution. 

In lTuite et al.l ( 20ld ). a log- normal distribution and WeibuU distribution were fit to both data 



sets. To estimate the densities the authors used Excel's solver. After assessing goodness-of- 
fit, the final model chosen was the log- normal distribution. In Figure [8] (left) and Figure [9] 
(top), we show the log-normal fitted distributions and compare it with the log-concave MLE. 
Pointwise 95% confidence bands based on the MLE are also shown. It is easy to see that the 
log-normal does not capture well most aspects of the empirical distribution. We make the 
following notes about the log-concave MLE. 

• Both the incubation data and duration data have been grouped, and therefore a dis- 
crete/grouped model is more appropriate than a continuous one. 

• The MLE captures well the shape of the empirical distribution, including the mode 
and the height of the mode. Notably, the MLE has the same mean and range as the 
empirical distribution. As shown in (|3.6p . the MLE will have a smaller variance than 
the empirical distribution. 

• Having estimated the MLE it is very easy to sample from its distribution. Given the 
seemingly accurate fit of our new nonparametric estimates compared to the empirical 
pmf, as demonstrated in Section [U we argue that these random numbers would be more 
accurate than those from the log-normal model. 

• Log-concavity encompasses many parametric models, but is substantially more flexible 
than any particular model and can capture a wide range of possible shapes. Moreover, 
the MLE is fully automatic, as it does not necessitate a choice of kernel, bandwidth, 
or prior. In this example, the MLE fits the empirical well, and it also "smooths" the 
empirical especially in the rather variable tail of the symptom duration distribution. 

• In the analysis of an infectious disease, the incubation period is of great importance, 
particularly so in the lower tail of the distribution, as this provides information on the 
rate of spread of the virus within a population. The log-normal does not fit the lower 
tail of the empirical distribution, and the MLE is better at describing this behaviour. A 
closer examination of the empirical data shows a spike at zero, which is most likely caused 
by inaccurate reporting of the onset of symptoms. To better describe this behaviour, 
we also fit a mixture of a log-concave pmf with a point mass at zero. This is, essentially, 
a zero-inflated log-concave distribution. To perform the fitting procedure, we used the 
EM algorithm. The results are shown in Figure [8] (right). The mean of the pure MLE 
was 3.88, which is equal to the mean of the data. The mean of the log-concave part of 
the mixture model was slightly higher, at 4.02. 
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Figure 9: Estimates of duration for HlNl data. 
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The data for the duration of symptoms of the swine flu has a clear spike at t = 7 
days. As above, this is probably caused by mis-reporting, as seven days is equivalent to 
one week, and therefore a likely choice in a patient's response. One ad-hoc method to 
account for this, is to again fit an inflated model, this time placing the mass function 
at t = 7 days. The results are shown in Figure [D (bottom) . The mean of the pure MLE 
was 8.66, which was also the mean of the fitted mixture model. The mean of the log- 



20 



concave part of the mixture model is slightly higher, at 8.72, however, the log-concave 
component has a lower mode at t = 6. The probability of observing an "inflated" value 
at seven was found to be 0.031. 

In addition to the aforementioned is sue, it is quite likely tha,t the d uration data collected 
suffers from length-bias (see e.g. lAsgharian and Wolfsonl . |2002| ). in that those with 



longer duration of symptoms were more likely to be observed. It would be of interest 
to see if our methods can be modified to include a length-bias correction, however, this 
is beyond the scope of this work. 

7 Discussion 

Fitting discrete data that exhibit unimodality is one motivation of using the log-concave 
MLE. Assuming that such data are drawn from a log-concave distribution is an attractive 
solution in the absence of an explicit characterization of a nonparametric unimodal estimator 
or a good justification of some parametric model. As was proved in Section HI the log-concave 
MLE will converge almost surely to the true distribution when this distribution is in fact log- 
concave. Furthermore, the estimator will still converge to something m eaningful in case the 



assumption of log-concavity does not hold. These results parallel those of lCule and Samworth 
(|20ld ) for the log-concave MLE on M^. 



In Section [4.21 we establish pointwise asymptotic theory for the MLE. The limit is described 
via a process that acts as an envelope of a finite dimensional Gaussian process on any given 
finite segment on which the true pmf is log-linear. The assumption that the log-pmf admits 
knots which are a finite distance apart is crucial for the derivation of the asymptotic distribu- 
tion. Hence, the geometric and double-geometric distributions are excluded, as is any part of 
a log-concave distribution which is log-linear on an infinite set of integers. Showing existence 
of the envelope process involved in the limit of the log-concave MLE (and hence existence of 
this limit) relies on showing that the solution of a well posed least squares problem is fully 
characterised by the properties of this e nvelope. As already noted abo ve, our approach is 



much i nspired by the pioneering work of iGroeneboorn et al.l (l2001all and iGroeneboom et al 
( 2001bl ) in convex estimation, and also by the work of lBalabdaoui et al.l (|200 y) in log-concave 



estimation for absolutely continuous distributions. The main idea, which had to be re-adapted 
to the discrete setting, is that the limit inherits the same (characterizing) properties of its 
finite sample counterpart. It is the "linearisation" of the shape constraint which transforms 
log-concavity to concavity that naturally gives rise to the relevant least squares problem. In 
a sense, maximization of the log-likelihood for a finite sample has been transformed into min- 
imization of a least squares criterion for the "infinite" sample. It is of much interest to note 
that some features are intr insic ally common to th e asym ptotic theory developed here and in 
Groeneboom et al.l (l2001bl ) and iBalabdaoui et al.l (120091 ). Others are evidently more specific 



to each setting. This is summarised in Table [T] where we compare the results for log-concave 
estimation. We use the usual notation HI and Y for the envelope and Gaussian process, and 
ip for the logarithm of the density in both discrete and continuous settings. 

It is well-known that in the context of shape-constrained estimation, the asymptotic behaviour 
is different in the "strict" and in the "degenerate" case. For example, a decreasing density 
satisfies f'{x) < 0. For the Grenander estimator of a decreasing density, if the true density 

21 



satisfies /'(xq) < 0, then we observe an n^'^ rate of convergence ( Groeneboonj . Il985l ). This is 
the "strict" case. The de generate case occurs when f '(x) = over some region, and here the 
convergence is of rate ^/n ( Carolan and Dvkstral . ll999l ). Furthermore, the hmiting distribution 
is different in these two cases. For other shape-constraints on M, only the asymptotic behaviour 
in the strict setting has bee n estabhshed, to our kn owledge. In particular, for estimation of 



a log-concave density on R, balabdaom et al.l ») showed thai if ^(x) = log /(.) satisfies 



^'"(xo) < 0, then pointwise convergence occurs at a rate of n?'^. Asymptotic behaviour when 
ip"{x) = over some region is still an open problem. 



continuous 



discrete 



^"(x) < 



Rate of convergence 



Y 



n 



2/5 



M" is concave 

involves integral 

of Brownian motion 



(AV;)^ < (A^)^ = 

{AM)x/px is concave 

involves cumulative sum 

of Brownian bridge 



Limit of the MLE at x involves EI"(0) (AM)^. = W(x) (AM)^ 



Table 1: Comparison of components of limiting theory for log-concave estimation of a proba- 
bility density and a pmf. 

Our results establish limiting distributions in the strict ((Ai/;)^ < 0) and degenerate {{AiIj)x = 
0) cases for a finite region. As such, they give insight into the asymptotic behaviour in the 
degenerate case in the continuous setting, and we conjecture that the limiting distribution 
will be characterised by a solution to the LS problem 



argmiUg concave on {a,b) 



h{x)g^{x) 



g{x)dV{F{x)) 



where /o is the true log-concave density, U denotes again the Brownian bridge, and (a, h) is 
the largest interval where ■(/'" = . 

We also note that our results are similar to those of IJankowski and Wellnerl ( 20091 ) where the 
Grenander estimator for the discrete decreasing pmf was studied. There it was shown that 
when (Vp)a; < over a region, then the MLE is asymptotically equivalent to the empirical 
pmf, and if {S/p)x = over a region, then the MLE's limit is described in terms of a LS 
problem. For the decreasing pmf there does not exist a pmf with {Vp)x = over an infinite 
region, and therefore the infinite region problem is unique to the log-concave setting. 

For the Grenander estimator. Ijankowski and Wellneii ( 20091 ) also describe global convergence 
rates for the MLE estimator. However, the problem there is considerably simpler than for 
the log-concave MLE. Without going into too much detail, for the Grenander estimator the 
limit is described in terms of the underlying process W(x) = \]{F{x + 1)) — 1LJ(-F(x)), which 
is well-defined. On the other hand, the limit for the log-concave MLE is describ ed in terms 
of th e underlying process W{x)/^/p^, which is not well-defined globally (see e.g. IVaradhanl . 
19681 ). Therefore, it appears that more subtle technical tools will need to be developed to 



study global convergence in this setting. We intend to explore this problem in a future work. 
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Appendix A 

Throughout the paper we use the following notation: 

• the words sequence (indexed by Z) and function (defined on Z) interchangeably. In the 
first case, the k-th term of a sequence p will be denoted by pk and the whole sequence 
by {pfc, k G Z}. In the second case, the value of the function p : Z — t- [—00, co) at A: G Z 
will be denoted by p{k), 

• C to denote the class of all concave functions ^ : Z — )• [—00,00), 

• CCi to denote the class of all log-concave probability mass functions, 

• CC to denote the cone of all log-concave, non-negative sequences indexed by Z; i.e. the 
set of all non-negative sequences p such that pk > 0,k £ Z, and satisfying the properties 
A and B of Definition 12.31 below. 



• z+ = l2>0, 

• 6ij = 1 if i = J, and otherwise, 

• 4(p, q) for the distance iJZk&^Pk - Ik)'') if 1 < /c < 00, and sup^g^ \Pk-qk\ if A: = 00 
for two probability mass functions {pk, k £ Z} and {qk, k £ Z}, 

• Ti for the Hellinger distance, that is, 'H'^{p,q) = 2^^ J2keziVPk ~ \/9fc)^' 

• For a sequence {ipx,x £ Z}, {Vip)x = ^x+i — ^x denotes the discrete gradient, and 
{Aip)x = ifx+i — 2ipx + <^x-\ denotes the discrete Laplacian. 

Appendix B 

In this section, we provide a proper definition of discrete log-concavity and provide a discussion 
of some parametric distributions that are log-concave. 

7.1 Discrete concavity and log— concavity 

Consider a function / : Z — )■ [—00,00) such that / 7^ —00. We will say that / is concave if 
the piecewise linear function / : M — >■ [—00,00) defined as 

fix) = (A; -|- 1 — x)f{k) + (x — k)f{k + 1), for x £ [k,k + 1] and for all k in the domain 
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is concave, in the sense that 

sub(/) = {ix,f,)eR^:fix)>fi}, 



the subgraph of /, is a convex set in M^; see iRockafellarl (jl970l ). Let dom(/) denote the 
effective domain of /, that is, 

dom(/) = {x G M : 3^ G M such that (x,/i) G sub(/)} = {x G M : f{x) > -oo}. 

Then it is easily seen that doni(/) is convex, and hence an interval in M. Also, concavity of / 
is equivalent to concavity of its restriction on doni(/). Note that excluding the value oo from 
the set of values of / avoids us having to deal with the undefined expression oo — oo. Also, 
this makes / a finite concave function on dom(/), taking possibly — oo outside this domain. 
Now, if we define the effective domain of the original function / to be 

dom(/) = {keZ: f{k) > -00} 

then concavity of / implies that dom(/) is a convex set in Z. Indeed, we can write 

dom(/) = M < [A;, /c + 1] such that k and A; + 1 G dom(/) >. 
fcez '^ ^ 

It follows that dom(/) is an interval in R if and only if dom(/) is a set of consecutive integers; 
i.e., is convex. Finally, the function / is concave on Z if and only if its restriction on dom(/) 
is concave. This follows easily from noting that concavity of the restriction of / on dom(/) is 
equivalent to concavity of / on dom(/). 

Hence, to construct a concave function / on Z, it is necessary and sufficient to consider a 
function that is finite and concave on some nonempty subset S of consecutive integers and 
extend it to a concave function on Z, in case 5 7^ Z, by setting / = —00 outside S. 

It follows that given a convex subset S := {A;i, ..., ^2} of Z, a function / : Z — ;• [—00,00) 
such that / is finite on S and / = — ooonZ\5is concave if and only if the left and right 
derivatives of / on the real interval {ki,k2) are nonincreasing. This is in turn equivalent to 
having nonincreasing order of these derivatives for two adjacent sub- intervals [k — l^k] and 
[k,k + 1]; i.e., 

f{k + 1) - f{k) < f{k) -f{k-i), ke {ki + 1, ..., k2 - 1} 

or equivalently 

;(.,>/(t^I)±Z<^±i)..,Z ,7,1) 

using the conventions —00 + c = —00 for c G M U {—00} and —00 > —00. Property 17.11 is 
usually referred to as mid- concavity. 

In conclusion, a function / : Z — t- [—00,00) such that / 7^ —00 is concave if and only if there 
exists a nonempty set 5 of Z of consecutive integers such that 

1. / is finite on S, 
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2. / = — oo on Z\S, 

3. / is mid-concave on Z; i.e., satisfies the property in (I7.ip . 

The definition of concavity of a sequence p = {pk , A; G Z} is straightforward and follows from 
the identification of the terms pk with the value at /c of a function defined on Z. 

We can give a natural definition of log-concavity of a sequence p = {pk ,k £ Z} such that 
p < oo. Such a sequence p is said to be log-concave if logp = {logpk, k £ Z} is concave, that 
is, there exists a nonempty set S of consecutive integers such that 

1. logpk > — oo, for k £ S, 

2. logpk = — oo, for A; G Z \ 5, 

3. log PA: > 1/2 (logpfe_i + logpk+i), for all k £ Z. 

This corresponds exactly to Definition 12.31 whose Property A is equivalent to 1 and 2, and 
Property B to 3. 

7.2 Some examples of log-concave pmf s 

Let n,r £ N*, a £ (0,1) and 9,iJ, £ (0, oo). The following parametric models are some 
examples of discrete distributions admitting a log-concave pmf. 

1. A binomial distribution with pmf 

/(^)a^(l-a)-^ k£{0,...,n} 
Pk = \ 

I 0, otherwise 

is log-concave with bounded support S = {0, ...,n} and a ratio sequence {pk/pk~i,k £ 
{0, ...,n}} = {oo,na(l — a)"-*^, ..., {n—k+l)k~^a{l—a)~^, ...,n~^a{l—a)~^}. The mode 
of p is located at the smallest integer /c G {0, ...,n} such that (n — A;+l)fc^^a(l—a)^^ > 1 
and (n - k){k + l)~^a{l - a)~^ < 1, that is, k £ [{n + l)a - 1, (n + l)a]. 



2. A negative binomial with pmf 



/k+r-l^^^^_^y^k^ ^g]N 



Pk = { 

I 0, otherwise 

is log-concave with unbounded support 5 = IN and a ratio sequence {pk/pk-i,k £ 
M} = {oo, ar, . . . , a{k + r — l)/fc, . . .}. he mode of p is located at the smallest integer 
k £ {0, ...,n} such that a{k + r — l)/k > 1 and a(k + r)/(k + 1) < 1, that is, k £ 
[{ar - !)/(! - a), {ar - 1)/(1 - a) + 1]. 

3. A geometric distribution with pmf 

fa(l -«)'=, fceN 
lo, otherwise 

is log-concave with unbounded support (from one side) 5 = N and, except for the first 
term, a constant ratio sequence {pk/pk-i, k £ N} = {oo, 1 — a, ..., 1 — a, ...}. The mode 
of p is located at 0. 

25 



4. A Poisson distribution with pmf 



Pk 



e-^e^/kl, A; G N 
0, otherwise 



is log-concave with unbounded support (from one side) 5 = N and a ratio sequence 
{Pk/Pk~i,f^ S N} = {(X),9,6/2, ...,6/k, ...}. The mode is located at the smallest integer 
A; G N such that k e \6 -1,9]. 



5. A Skellam distribution with pmf 



Pk 



oo 



0k+ 



m ..m 



^-^ mUk + m)\ 

m=0 ^ ' 



k€ 



is log-concave with unbounded support 5 = Z. Log-concavity of the Skellam distribution 
is a consequence of the fact that it is the distribution of the difference of two independent 
Poisson random variables, here with parameters 9 and /i. Indeed, it is not hard to 
see that if X is a log-concave random variable, then so is —X. The claim of log- 
concavity follows by appealing to c losedness of the class of l og-concave distribution 
under convolution (see Theorem 2 of lKeilson and Gerberl . ll97ll ). For A: £ Z, we have 



Pk 
Pk~i 



where Im. is the modified Besse l function of the first kind and order m (see e.g. 




Abramowitz and Stegunl.ll964l . Section 9.6). The Skellam distribution was originally in- 
troduced bv lSkellamI (j 19461 ) and found applications in many fields, such as modeling the 
difference of the number of goals in football games, and the intensity difference of pixels 
in cameras. For more detail s about this distributio r i, we r efer the reader for exa r nple t o 
Karlis and Ntzoufrasl (fJoO^) . iKarlis and NtzoufrasI (IpOOSl'l . IXarlis and Ntzoufrad (120061). 
Karlis and Ntzoufraa ( 20091 ). and lAlzaid and Qmaiii ( 20ld ) and the references therein. 



Appendix C 

This section contains all proofs and relevant technical details. 
7.3 Proof from Section [2] 



Proof of Proposition \2.SX The result follows immediately from iDharmadhikari and Joag-Dev 
(Il988l . Theorem 2.8) since then 



P{X(^A,f > P{X € Ai+i)P{X e Ai^i), 
from which it follows that {pi} is log-concave. 



D 
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7.4 Proofs from Section 13.11 

Proof of Theorem \3.1[ We give first a proof of (i) . Let </? be a concave function in C such that 
^n(v') > — oo. Let now ip be the unique concave function in Cm(I) such that V'(fcj) = '^(ki) 
for i S {l,...,m} and interpolating between the observations, that is if kt < /cj+i for some 
i G {1, ..., m — 1}, then for any k G {ki + 1, ..., A;j-|_i — 1} we have 

m = m) + ^^'T^~1}^'\ k-h). 

Since ip is concave, we have the fohowing properties: 

1. tp <ip onZn [ki,km], 

2. $n(v^) > ^n{^) with equahty if the two functions coincide. 

This imphes that in maximizing the criterion $„, the latter does not decrease when we replace 
a function in C by a candidate in the class Cm(X). Hence, we can restrict attention to functions 
in the latter class which formally means that if a maximiser of $„ exists, we should look for 
it in the smaller class Cmi^)- 

Next, we prove (ii). Let (V'^)p be a maximizing sequence of the criterion $„ in 6^(1)', i-e., 
limp_!.oo ^nCV'^) = s^Pip&Cm(i) ^n{ip)- We will show now that the maximization problem can 
be restricted to a compact subset of Cm{I)- 
Consider the functions fj{x) = WjX — exp(x), j = 1, . . . ,m. It is easy to see that 

m 

<^n{r)<^fj{r{kj)). (7.2) 



Now, suppose that there exists j E {1, . . . , m} such that \ip^{kj)\ — )• oo as p — )• oo. From (j7.2p 
and coercivity of fj, it follows that $„(V'^) — t- — oo which yields a contradiction since (V'^)p 
is a maximizing sequence of $„ (we should have at least limp^oo ^(V'^) > —{km — ki + 1), 
the value taken by the criterion for the sequence whose all terms at equal to zero.) Hence, 
there exists K > such that for all j G {1, . . . ,m}, |^p(/cj)| < K. Since V'^ is linear between 
the observations kj,j = l,...,m, it is easy to see that this bound actually applies for all 
k £ {ki,ki + 1, . . . ,km — ^,km} = 'Zf^[ki, km]- It follows that the closed subset (concavity is 
defined by a set of non-strict inequalities) 



\tp:-ipe Cm{I) and max \i;{k)\ < k} 
I- fceznffci.fc^l J 



fcGZn[fci,A;m] 

is bounded, and hence compact. Continuity of $„ concludes the proof of existence. Uniqueness 
follows from strict concavity of $„. D 

7.5 Proofs from Section 13.21 

In the sequel, we will make extensive use of suitable perturbation functions; i.e., functions 
which determine an admissible direction in which the MLE is perturbed. All these results 
will be based on the following proposition, which gives suitable directional derivatives of ^n- 
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Proposition 7.1. Let cq > 0. Consider functions Pi and P2 defined on TL such that ipn+^Pi £ 
C for all e G (0, eo) and ipn + ^P2 G C for all e G (— eo) eo)- Then, 

and 

TYh rim 

Y (Mkj) - Mkj-i)) P2{kj) = Y i^r^ik) - Fn{k - 1)) P2ik). (7.4) 

j=l k=ki 

Proof. Using the convention F„(/co) = Fn{ko) = and the fact that ipn is the maximiser of 
<&„, we can write 

U > lim 

£\o e 

m km 

= Y'^jP^ikj) - Y ewi^n{k)Pi{k) 

j=l k=ki 

m km 

= Y (^"(^i) - ^nikj-i)) Pi{kj) - Y [Fn{k) - Fn{k - 1)) Pi (A:) 
j = l k=ki 

where we take ko = ki — 1. This yields (|7.3|) . To get (|7.4|) we use the fact that the Hmit above 
is equal to zero when replacing Pi by P2. D 

Proof of Lemma \3.SX First, we assume that ij: = ij:. For x G Z n [fci, km\, consider the pertur- 
bation function 

P,(A;) = -{x-k)+. (7.5) 

Then, for all t > the function tp + tP^ is concave. By Proposition 17. H we have that 

Tfl t^m 

> - ^ {¥n{kj) - F„(A;,-„i)) (x - A;,-)+ + ^ (i^n(fc) - Fn{k - 1)) (x - k)+. 

j=l k=ki 

It follows that 

m—l 
> -¥n{km){x - km)+ - ^ ¥n{kj) {{x - kj)+ - {x - fcj+l) + ) 

i=i 

+ Fnikm)ix - km)+ + J^ ^"(^) ((^ " ^)+ " (^ " (^ + 1)) + ) 

fc=fci 

= Y ^nikj){k,+i - k,) + F„(fc,J(x - A;,-J + Y Pn{k) 

j=l k=ki 



28 



since ¥n{km) = Fn{km) = 1 and for any integers r < s 

{x — r)+ — (x — s)+ = s — r if X > s 

= X — r \{ r < X < s 
= otherwise. 

Here, jx is defined to be the unique index such that kj^ < x < kj^^i. Hence, 

jx-l X-1 

^ ¥4k,)ik,+i - k,) + F„(fc,J(x - k,J > Y^ Fnik). 

j=l k=ki 

If X is a knot point, then for \t\ smah enough the function tp + tPx is concave. By Proposition 
17.11 it follows that for such an x the condition in (|3.4p is satisfied. 

Conversely, suppose that V' satisfies the conditions of Lemma [3.2[ Let ■0 be a concave function 
in Cm{I)- By (strict) concavity of <J>„ and using the same summation by parts as above, we 
have 

^n{i')-^nW > Y.Wj{i>ikj)-i;ikj)) - J2 exp4>{k){i>{k)-^P{k)) 
j=l k=ki 

m—l 
= Yl ^nikj) (i^ikj) - i^ikj) - Wkj+^) - V(%+l)' 

i=i 

- J2 ^«(^) (^(^) - ^(^) - (^(^ + 1) - ^(^ + 1) 

k=ki 
m—l km — 1 

= Y ^n{kj) [i'ikj) - i'ikj+i)) - Y Fn{k) (^(fc) - i^ik + 1)] 
j=l k=ki 

m—l km — l 

Y Vnikj) {^{kj) - V(%+l)) - Y. Pn{k) mk) - ^{k + l))\. 

j=l k=k\ 

Now, using ()3.2p . both conditions ()3.4p and (j3.3p and the fact that q < we have 

m—l km — l 

Y Mkj) (Hk,) - Hkj+i)) - Y ^n{k) Wk) - ijik + 1)) < 0. 

j=l k=ki 

Indeed, for i = 1, ...,p we have by the calculations above and taking x = ai 

m—l km — l 

Y ^n{kj) {{a, - kj)+ - {a, - %+i)+) - Y ^"(^) (("'• - ^)+ - («i - (^ + 1))+) ^ 0- 

jf=l k=ki 

On the other hand 

m—l km — l 

Y ^n{kj) {{a + bkj) - (a + bkj+i)) - Y ^n{k) {a + bk-{a + b{k + 1))) 

j=l k=ki 

{m—l km-1 I 

Y Mkj){k,+, - k,) - Y Fnik) \ = 
j=l k=ki ) 
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which follows from p.4p for the knot point k^- Finally, 

rra— 1 km — l 

Y, Mkj) (i^ikj) - i^{k,+i)) - Y, Fnik) [m - ^{k + 1)) = 

j=l k=ki 

using (|3.2p and the equality condition in (|3.4p for the knot points of if) (including km)- It 
follows that ^n{i>) > ^n(^), and that ijj = tp. D 

Corollary 7.2. Suppose that n is a double knot of the log-MLE. Then Fn^n) = F.„(k). If k 
is a triple knot of the log-MLE, then pn,K = Pn,K.- 

Proof. From Lemma 13.2^ we have that for any knot point k the equality 

k<K—lx<k k<K—lx<k 

holds. If K is a double knot, then the above equality holds at both k and k + 1. Taking 
differences, yields that 

^n{K) = Y^Pn,x- Yj X]^"'^ = X]X]^"'^~ X] ^Pn,x = '^nin). 
k<KX<k k<K—lx<k k<KX<k k<K—lx<k 

If K is a triple knot, then the first equality holds at both k — 1,k and k + 1. Hence, 

Pn,K = Fn{K) - Fn{K - 1) = F„(k) - F„(k - 1) = Pn,«;- 

D 

7.6 Proofs from Section 14.11 

We begin with the following lemma. 

Lemma 7.3. Let pn,p denote probability mass functions on Z. The following statements are 
equivalent. 

• Pn,x -^ Px for all X eZ 

• ^k{Pn,p) — ^ for any 1 < k < oo 

• n{pn,p)^o. 

Furthermore, ifpn is log- concave for alln, thenp is also log-concave, and the above statements 
are also equivalent to 

• There exists an uq > 0, which depends on the pmfp, such that "^^^ \Pn,k — Pk\ — ^ 0, 
for all a G [0, oq]. 



Note that the second part of the lemma is essentially Proposition 2 of ICule and Samworth 



( 20101 ) in the discrete setting. The proof in our case is, naturally, considerably simpler. We 



leave the proof of this statement to the Appendix. 
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Proof of Lemma \ 7. 3\ Suppose first that Pn converges to p pointwise. Fix e > 0. Then there 
exists a K such that Y2\x\<kPx > 1 — e/2. Furthermore, there exists an no such that 
sup|2.|<;^ \Pn,x — Px\ < e/2(2K + 1), which imphes that 'Yli\x\<KPn,x > 1 — £• It follows that 

y^ \Pn,x -Px\< ^ \Pn,x -Px\+ ^ Pn,x + ^^ Px < ^E. 
X \x\<K \x\>K \x\>K 

Therefore p„ — >• p in ^i. However, since Pn^P are also elements of it for all 1 < A; < oo, it also 
holds that p„ — )• p in (.^ for any 1 < /c < oo. For convergence in Hellinger distance, pn^ we 
recall that pjjip, q) < 2~^pi{p, q). Finally, note that convergence in pk or pn implies pointwise 
convergence. This proves the first part of the lemma. 

To prove the second part, recall that if p„ — t- p pointwise then there exist random variables such 
that Xn =^ X and Xn, X have distributions p«, P- It i s know n that log-concavity is preserved 
under weak limits fsee iDharmadhikari and Joag-DevI ( 19881 . Chapter 4)), and therefore p is 



log-concave. Furthermore, Pn ^ P pointwise. 

Next, let ifn^k = logPn.fc and similarly ipk = logp^. It follows that ipn,k ~^ fk pointwise, and 
therefore also uniformly for k bounded. Without loss of generality, assume that is in the 
support of the mass function p. Also, there exist 6 S M and c > such that pk < e~'^'^'^^ for 
all k £ Z. Let ao = c and define e = (ao — a)/4 > 0. Then 

— I^n — < -ao + {b- ^o)/\k\ < -qq + e 

for all \k\ > R for some sufficiently large R. 

Since ipn is log-concave, it follows that {(pn,k — Vn,o)/\k\ is decreasing in k for \k\ > R, as long 

as R was chosen large enough. Hence, we also have that 

Vn,k - Vn,0 . , „ 

-^\ ^-"0 + 2^ 

for all I A; I > R and all n sufficiently large. We have thus shown that 

Ee""W < j; e-l'^ W- + 5] e°l'=l+^-«+(-"«+2e)|fe| 

k \k\<R \k\>R 

|A:|<_R |fc|>_R 

from which it follows that ^^ ^'^^"pn.k is summable. The result now follows from the domi- 
nated convergence theorem. D 



Proof of Theore7n \4.1\ The proof is divided into several steps. First, consider the pmf q 
proportional to e"!*^!' Then pxL {q\\ p) is finite by the assumptions of the theorem, and hence 
there exists a sequence of log-concave probability mass functions qn such that pxL {q-n \\p) ^ 
argmiUgg^CiPKL {q \\ p) ■ Since pkl {qn \\ p) < Pkl (qW p) , it fohows that 



SUp> I -logqn,x\Px < V(-logg'a;)pa; < OO, 



31 



which in turn imphes that for any fixed m 

sup ^ \-logqn,x\ < sup — y^(-logg^)pa; < oo. 

n I 1^ \x\<mPx ^™ 

Therefore, for each m, there exists a 6 > such that inf„ inf |2,|<^ Qn^x > 5. If M is sufficiently 
large, it follows that limsup„ supi^i^ji/ gn^^,' ^ ^/2, which in turn implies that there exists a 
function h^ = —a\x\ + /3 such that sup^^Qn^x < e'*'"- 

Let X„ denote the random variable with pmf qn- It follows from the above that X„ is tight, 
and therefore qn has a convergent subsequence. Let qo denote its log-concave limit. By Fatou's 
lemma 

PKL {qo lb) < liininf pkl {qn \\p) = argmin^g^CiPKL {q\\p), 

which shows that a minimiser exists. 

Next, suppose that pi and p2 both minimise />kl (• || p) , and let p be the pmf proportional to 
(piP2)^/^. Then 

V'plog- = V'plog— + log ^^(^1^2)^^^ < V'plog — 
^-^ p ^-^ p\ ^-^ ^-^ p\ 

by the Cauchy-Schwarz inequality, with equality if and only if p\ = P2- Therefore, the min- 
imiser is also unique. 



It remains to show that pn — )• p- Recall that pn satisfies the inequality in (13. 6p . Let Xn denote 
a random variable with pmf p. Then, by Markov's inequality, 



P(\XJ >m)< 



l^xeZ l^lPn,: 



m 



Therefore, Xn is tight, and hence there exists a subsequence of pn, which we denote again 
by {n}, such that pn — s- p, for some pmf p. From Lemma 17.31 it follows that p must also be 
log-concave. It remains to show that p = p to complete the proof. 
By definition of the MLE, we have that for any 6 > 

< y2log{b + Pn,k) Pn,k -y^^Og PkPn,k 

= X] ^°S(^ + Pn,k) {Pn,k - Pfc) + X logPfc {Pk - Pn,k) 

, ST^, fb + pn,k\ , Y^, fb + pk\ 

We next get rid of the first two term on the right-hand side. First, using summation by parts 

'^log {b + pn,k) {Pn,k -Pk) = X(F„(A;) - F{k)) [log(6 + p„,fc) - log (6 + p„,A:-l)] , 

k&z fcez 
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and hence 

^ log {b + Pn^k) {Pn,k - Pk] 



< sup|F„(A;)-F(A;)| J V [log (& + p„,fc) - log (6 + p„,fc_i)] 

I k<m 

+ ^ [log {h + p„,fc) - log {h + p„,fc-i)] > 

k>fh ) 

< 21og(5 + p„,^)sup|F„(A;)-F(A;)| 

fcez 

< 21og(l + 6)sup|F„(A;)-F(A:)|, 

feez 

which converges to zero. The law of large numbers shows that the second term also converges 
to zero. Therefore, 

limsupVlogf — — r^ ) Pfc < V log ( /"^ ] Pk 

n ^-^ \b+Pn,kJ ^-^ \b + PkJ 

which implies that (using Fatou's lemma) 

lim sup lim sup 7 log I ;^— ) Pfc < 

6^0 n ^-^ \b + Pn,kJ 

Taking limits (by the dominated and monotone convergence theorems) yields that 



or, in other words. 



E>°<|)- . E.o.(|) 



Pk, 



which, as p is the unique minimiser of the quantity on the right hand side, proves that 

p = p. n 

Proof of Corollary \4-^ This follows since 



\Fn{x)-F{x)\ < ^\Pn,y-Py\ < h{Pn,P)- 
y<x 



U 



Proof of Lemma \4.3[ A point r is a knot of ipn if) and only if, (A^„)s < 0. Let e = — (A'(/')a; > 
0. It follows from Proposition 14. H that there exists an tlq such that for all n > tlq 

max \tl;n,x - ipx\ < e/8, 

x=r—l,r,r+l 
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with probability one. Therefore, 



< {Aip)r + 4 max IV'n.x - ipx 

\x=r—l,r,r+l 

< -e + e/2, 

which is strictly negative, as required. D 

7.7 Proofs from Section 14.21 

The following proposition is crucial for establishing the weak convergence of the MLE. 

Proposition 7.4. Let r < s denote two successive knot points. Then for all x £ {r, . . . , s — 1}, 

^ {'ipn,x — i^x) CLiT-d y/n {pn,x — Px) CLf^ bounded in probability for all n. 



To prove the above proposition, we start with the following lemma. 
Lemma 7.5. If r is a knot of ipn then 

Pn,r > Pn,r, (7.6) 

Proof. Let q denote the first knot point before r. From Lemma 13.21 we have the following 
statements 

r r 

J;F„(x) > ^F„(x) (7.7) 

x=q x=q 

r—1 r—l 

^F„(x) = Y.^n{x) (7.8) 

x=q x=q 

T-2 r-2 

Y^Mx) > 5]i^n(x), (7.9) 

x=q x=q 

where Fn denotes the cdf of the MLE. Taking (USD - 2 (USD + dZSD yields (USD- □ 



Proof of Proposition \7.4\ Showing boundedness in probability of \/n{pn — p) on {r, . . . , s — 
1} for any consecutive knots of V = logp is equivalent to showing the same property on 
{0, . . . , r — 1} for any knot r occuring the left of zero. If r = 1, then Lemma 13.21 implies that 
Pn,Q = Pn,o and boundedness immediately follows at = r — 1. From now on, r > 1 will be 
assumed. Let < u < v < r he consecutive (random) knots of ipn such that {u, ■ ■ ■ ,v} is the 
first (random) segment containing at least one interior point; i.e., v > u+1. Then, all knots 
occuring before u are triple knots and by Corollary 17.21 it follows that 

sup Vn\pn,k-Pk\= sup Vn\pn,k-Pk\ < sup Vn\pn,k-Pk\ = Op{l). 

0<k<u-l 0<k<u-l 0<k<r~l 
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We will now show that sup„<fc<j,_i i/n|p„^fc — Pk\ = Op{l). Let S > 0. For a given x G 
{u, . . . ,v — 1}, consider the perturbation function au,v,x, where 

\^, ifk(^{x + l,...,v-l}. 

If 6 is chosen to be small enough, then it is not difficult to see that ip + 5au,v,x G Cmi^)- 
Applying Proposition 17.11 yields 

1"— 1 v—l 

y^ Pn,kOiu,v,x{k) < ^ Pn,kOLu^v,x{k)- 
k=u+l k=u+l 

Using Lemma 17.51 and the fact that both ip and tpn are linear on {u, . . . , v}, we can write for 
/c e {n + 1,... ,w- 1} 



v — k k~u 
v — u ^^v — u 



v — k k — u v — k k — u 



Pk — Pu Pv and Pn,k — Pn,u Pn,v S: Pn,u Pn,v 

It follows that 

v—l v—l 

y^ V^{Pn,k - Pk)au,v,x,k < y^ VniPn,k - Pk)au,v,x,k 

k=u+l k=u+l 

/ v — k k — u v — k k- 

< Yl ^[pnTuPn^ -K""K"" ) a«,^,x(fe). 
k=u+l ^ 

Since au,v,x < 1 this further implies 

v-l 



(7.10) 



1) inf \/n\pn,k-Pk\ < yi VniPn,k 



0<k<r 



Pk)C>-u,v,x,k 



k=u+l 

< ir-l] 



sup yn 

0<u<v<r,u+l<.k<.v-'l 



v — k k — u v — k k — u 

f^v-u-v-u _„i'-u„i'-u 

Pn,u Pn,v Pu Pv 



Using the central limit theorem and the delta method, it follows that 



v-l 

Y y/n{pn,k - Pk)au,v,x{^) = Op{l) 



(7.11) 

k=u+l 

for all a; G {u + 1, . . . ,v — 1}, and where Op(l) does not depend on u, v nor x. Hence, we have 
obtained a linear system, and (j7.1ip can be rewritten as 

A {Vnipn,u+i - Pu+i) ■ ■ ■ VniPn,v-i - Pv-l)) = (Op(l) . . . Op{l)Y 

where A is the [v — u — 1) x {v — u — 1) matrix (aM,i),x(^))u i k^ x<v-1' ^^ some algebra we 
can show that A is invertible such that 

/ 2{v-u-l) 2{v-u-2) g g \ 



A 



-1 



v—u 



v—u 



v-u-1 4{v-u-2) 

V- 









3{v—u—3) 








v—u v—u v—u 

2{v-u-2) 6{v-u-3) 4:{v-u-4:) 

v—u v—u v—u 

3{v-u-3) 8(t)-M-4) 

v—u v—u v—u 











5(d--u-5) 














g 2{v-u-2) 2{v-u-l) . 

v—u v—u / 
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Since {v — u — l)/(w — u) < r — 1, A~^ admits bounded entries which in turn implies that 
the \/n{pn^k — Pk) = Op{l) for all /c G {-u + 1, . . . , u — 1}. If v > u + 2, then boundedness in 
probability at u (and also at v) follows from the previous conclusion and applying the delta 
method on any pair of the identities given in (jT.lOp . If w = u + 2, then by Lemma [3.21 we have 
that 

v—l v—1 

k=u k=u 

or equivalently 

Vn{pn,u - Pu) = VniPn,u -Pu) + - [Vn{pn,u+1 " Pu+l) " Vn{Pn,u+l " Pu+l)] 

and boundedness at u follows by the central limit theorem and boundedness at u + 1 as 
shown above. The argument above can be repeated on the subsequent segments. Finally, 
boundedness of \/n{ipn — ip) follows by applying the delta method. D 



Proof of Proposition \4-4\ Existence of a minimiser follows from arguments along the same 
lines as in the proof of Theorem 13.11 (11) since the functions x i— )• — (x^/2 — x W{x)) are coer- 
cive for k = r, . . . ,s — 1. Uniqueness is a result of strict convexity of ^. 

Now, fix X G {r + 1, . . . , s — 1}. For t > 0, the function k i— )■ g*{k) + tPx{k), where Px is the 
perturbation function defined in (j7.5p . is concave and hence 

k=r 

Repeating the same calculation as in the proof of Lemma 13.21 shows that this is equivalent to 

x—l k 

Y.Y.^9*{y)Py-^{y))<^, 

k=r y=r 

which shows the inequality part of (|4.2|) . Proof of equality if x is a knot of g* uses the fact that 
the perturbation function Px satisfies that g* it tPx is concave for |t| small enough yielding 
limt_>o(*^(5* ~ tPx) ~ ^i9*))/t = 0. In this sense, s is always a knot point since the function 
Pg is simply the linear function on {r, r + 1, . . . , s — 1}. Therefore we always have equality at 
s, and also trivially at x = r by definition. It follows that the minimiser of ^ must satisfy the 
conditions (14.21). 



Conversely, suppose that g* is a concave function on {r, . . . , s — 1} such that the process H 
defined in ()4.3p satisfies ()4.2|) . Let g be a concave function on {r, . . . , s — 1}. We will show 
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that ^{g) > ^{g*). We have 

s-l s-l 

2 



^g)-Hg*) = lY.[9'(>')-(9*f{k)]pk-Y.l9{k)-g*{k)]W{k) 

k=r k=r 

^ s— 1 s— 1 

= 2 ^^^^^^ ~ 9*^^)fPk + E(5(^) - 9*{k)) {g*{k)pu - W{k)] 

k=r k=r 

s-l 

> Y.(9ik)-9*ik)){9*{k)pk-W{k)}. 

k=r 

Now, for a £ {r, . . . , s} 

s— 1 a— 1 fc 

J] {5*(fc)Pfc - W(A;)} {a-k)+ = Y,Yl i9*ik)Pk - W(A;)} = M{a) - Y(a) < 0, 

k=r k=r y=r 

with equahty if a is knot of g* , by assumption. Using a representation for concave functions 
on E {r, . . . , s — 1} similar to (|3.2p (note that the functions (a — k)^ have negative weights), 
it fohows that 

s-l 

j;5(fc){5*(A:)pfc-W(A:)} > 0, 

k=r 

with equahty if g is replaced with 51*. It follows that ^{g) > ^(g*) and hence that 5* is the 
minimiser of <5. D 

Lemma 7.6. The limiting distribution M also satisfies Ylx=r 9* (•'^)Px — Ylx'=r^i-^)- 

Proof. The proof follows from the argument above, choosing the perturbation function P{k) = 
±1. D 

Proof of Theore7n \4-5\ From the definition of the process and Proposition 17.41 it follows that 
EI„ is tight. Therefore, we apply Prokhorov's theorem, and it is sufficient to show that any 
convergent subsequence has the same limit, H. To do this, consider a subsequence of IHI„ 
(which we denote again by {n},) and let Q denote its weak limit. Note that all we need to do 
is to show that Q satisfies the condition in (j4.2p with equality at x = r, s, as these uniquely 
identify this distribution. 

We first prove the result on the first segment {0, . . . , s}. Lemma 13.21 implies that 

y-l k y-1 k 

^^Pn,x < ^Y^Pn,x, 
k=Ox=0 k=Ox=0 

with equality at all knots y, where s is always a knot. Subtracting from both sides X]fc=o Sx=o P^ 
and multiplying by y/n, yields that EI„(y) < Y„(y) with equality at all knots y. Note that in 
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the definition of Y„ and EI„ given above (and that of the related processes G„, W„, G„ and 
Wn) the knot r should be replaced by 0. Hence, 

y—l k y— 1 k 

Yn{y)>Mn{y) = y^ ^^ Vn(j)n,x - -Px) = y^ y^ \/ra(exp ijjn,x - exp ijj^) 

k=Ox=0 k=Ox=0 

y—l k y—l k 

= y^y^Px\/ra(V^n,x -i'x) + y^ y^ exp 0n,xVn{-tpn,x -1pxf/2, 
k=0 x=0 k=0 x=0 

for some On,y, by applying a Taylor series expansion of second order. The inequality holds for 
all X E {0, . . . , s} with equality at knots of ■0n- Note also that since ip is linear on {0, . . . , s}, the 
knots of ■0,1 are the same as the knots of ^/n{^l)n—tp) ■ Let Mniy) = X]fe=o Ylx=oPxV^i'^n,x—'4^x)- 
Then, by Proposition 17. 4|, 

Y„(x)+Op(l) > Unix) 

for all X = 0, . . . , s with equality at all knots of \/n{'il)n — ip) on {0, . . . , s — 1}. Note also that 
V^i^n — V') is concave on {0, . . . , s — 1}. 

It is well-known that Y„(y) converges in distribution to Y(y) for all y = 0, . . . , s, with Y G 
]^{''v,s} defined in Proposition 14.41 (for any successive knots r < s), and therefore so does the 
process Yn{x) + Op(l). Recall that Q denotes the weak limit of ]HI„, which is the same as the 
weak limit of ]HI„ = EI„ + Op(l). Taking limits in the last display above thus shows that 

Y{x) > Q(x), 

with equality at x = 0, s and at all knot points of the process {AQ)x/px- The latter follows 
from the fact that such a knot point is also a knot point of 

(Ain)x ^.7 ,. 

= Vn{ipn,x-V)- 

Px 

Therefore, Q satisfies the conditions in (|4.2|) . and the same boundary conditions at x = 0,s 
as H. Hence, Q = H, as required. 
Lastly, note that 

Vn{j)n,x-Px) = iAMn)x, 

and V^(F„(x) - F(x)) = (V]H„)^ 

since F„(— 1) = F„(— 1) = 0. This completes the proof on the segment {0, . . . , s}. 

Now, let s' > s be the next knot of the true log-pmf. The argument here is the same, except 
that we we no longer have that Fn{s — 1) = F„(s — 1) as we did for the knot x = 0. However, 
it is true that y/n(Fn{s — 1) — F„(s — 1)) — 7>p 0. This follows since 

V^(F„(s-l)-F„(s-l)) = ^/^{Fn{s-l)-F{s-l))-V^{¥n{s-l)-F{s-l)) 

^d H(s)-E[(s-l)-(Y(s)-Y(s-l)) 

= 5]<?*(x)p,-J^W(x) = 0, 

x=r x=r 

by Lemma 17.61 This allows us to repeat the above argument on the segment {s, . . . ,s' — 1}, 
and so on. Iterating in this fashion establishes the result in the general case. D 
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